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Abstract 

We show that strategies implemented in automatic theorem proving involve an in¬ 
teresting tradeoff between execution speed, proving speedup/computational time and 
usefulness of information. We advance formal definitions for these concepts by way 
of a notion of normality related to an expected (optimal) theoretical speedup when 
adding useful information (other theorems as axioms), as compared with actual strate¬ 
gies that can be effectively and efficiently implemented. We propose the existence of an 
ineluctable tradeoff between this normality and computational time complexity. The 
argument quantifies the usefulness of information in terms of (positive) speed-up. The 
results disclose a kind of no-free-lunch scenario and a tradeoff of a fundamental nature. 
The main theorem in this paper together with the numerical experiment—undertaken 
using two different automatic theorem provers (AProS and Prover 9) on random the¬ 
orems of propositional logic—provide strong theoretical and empirical arguments for 
the fact that finding new useful information for solving a specific problem (theorem) 
is, in general, as hard as the problem (theorem) itself. 

Keywords: experimental mathematics; automatic theorem proving; propositional 
calculus; tradeoffs of complexity measures; computational complexity; algorithmic com¬ 
plexity. 


1 Introduction 

Speed-up in automatic theorem proving should not be regarded just as a quest for faster 
provers. To prove a theorem is an essentially difficult task and a very sensible question is 
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whether there can be a shortcut to it by adding additional information. So the issue is 
what information can be added so that a proof is significantly shorter. Viewed from a very 
abstract point of view, of course, there is an easy (and trivial) answer: add the theorem itself 
and the proof becomes minimal! But adding all provable theorems to an axiomatic system 
renders it pointless. 

On the other hand, a system under memory constraints that are not minimal (i.e. that 
can store a set of useful theorems apart from the axioms of a theory) but at the same time 
bounded by a realistic measure, would be theoretically optimal if it stores only the most 
useful theorems for a certain task (that is, those that can make a proof shorter). A naive 
strategy would be to prove theorems incrementally and save them (until the memory is full). 
When a new theorem has to be proved the system can resort to previous theorems to see if 
a shorter proof can be obtained other than trying to prove the theorem from scratch. But 
how useful can this approach be? As we will show, it is not only naive and so expectedly 
not very useful, but it is also very rarely fruitful and in equal measure a waste of resources. 

For this, we use a very basic but objective measure of speed-up, namely shortening the 
length of proofs. We sample random propositional calculus theorems and with the help of 
two automatic theorem provers (AProS and Prover9) we produced alternative proofs for 
them including as additional axioms other (random) valid formulas. Afterwards, we compare 
the length of the proofs to establish if a shorter one arose from the use of the new axioms. 
In order to select our samples, we restrict our analysis to a fragment of propositional logic, 
given by an upper bound on the number of atomic propositions and of logical connectives. 

To find a proof for a theorem (3 from a finite set of formulas t using an automatic theorem 
prover, one can always start from t and apply all transformation rules until f3 is generated, 
then pick out the sequence on the path from t to (3 as proof. In practice, however, important 
optimization strategies have to be implemented to avoid exponential execution time even 
for the simplest of proofs. We will explore how implementing these algorithmic strategies 
leads to a compromise between various seminal complexity currencies, shedding light on a 
possibly more general challenge in problem solving, related to the usefulness of information 
and computational time. 


Definition 1. Let s be a finite set of formulas and let s b P stand for the fact that f3 is 
provable from s. D(s h (3) will denote the length of the minimum proof for s b (3, as given 
by the number of logical deduction steps used in the proof (for example, the number of lines 
in a Fitch diagram). 

Let t and t' be two representations of the class of equivalent theories [ t ] = [t'j (that is, t 
and t' are finite sets of axioms with an identical set of logical consequences, or theories) such 
that \t\ < \t'\. We define the speed-up delta for t b f3 between t and t' , or simply speed-up, as 
the function: 

D{t’ h P) 

D(t b P) ' 

Let P = {(pi} be a finite subset of the set of all valid formulas in propositional logic and 
let P G P and t C P. Now A will be the set of all logical consequences (theorems) of t in 
P. Finally, the set § is the subset of A such that for a given subset A of P , with P ^ A, we 
have the following property: 

8p(t,tU A) > 0. 
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Note that this property is equivalent to D(t h (3) > D(t U A b (3) 

We ask after the relation and distribution between the size of the set § with respect 
to the size of a finite subset of A as a function of the set A. In other words, given a 
random axiomatic system, if we strengthen it by adding a number of theorems as axioms, 
what does the distribution of non-trivial speed-ups look like? A set of valid propositional 
calculus sentences and axiomatic systems is constructed given three variables: n represents 
the maximum depth of composition of logical operators, m states the maximum number of 
different literals that can be present in a sentence, and j determines the number of axioms 
in an axiomatic system. 

We will report not only that instances of non-trivial positive speed-up are relatively rare, 
but that their number is considerably smaller than the number of instances of negative speed¬ 
up. In other words, strengthening a random axiomatic system by adding theorems as axioms 
tends to increase the length of proofs found by automatic theorem provers. We believe that 
the behavior observed verifies the stated condition and that its oblique distribution is strongly 
related to the computational difficulty of finding a proof of minimum size and deciding the 
usefulness of information as a whole. 


2 Methodology 

PROVER9 is an automated theorem prover for First-order and equational logic developed 
by William McCune mi at the Argonne National Laboratory. Prover9 is the successor of 
the Otter theorem prover. Prover9 is free and open source. AProS (Automated Proof 
Search) is a theorem prover that aims to fold normal natural deduction proofs of theorems 
in propositional and predicate logic [T8j . It does so efficiently for minimal, intuitionistic and 
classical versions of first-order logic. 

We undertake an empirical exploration of the speed-up in propositional logic, using 
AProS and PROVER9 in order to approximate the proof complexity over randomly gener¬ 
ated sets of axiomatic systems and propositions. We later propose a necessary but possibly 
insufficient property (in definition [7]) for judging the adequacy of the approximations so ob¬ 
tained. 

To perform the exploration described, we have first to narrow the search space: We 
denote the set of all propositions bounded by n and m by P(n, m), the set of all theories in 
P by T (T = 2 P ), while T(n, m) and A (n,m) denote the sets of all theories and arguments 
bounded by n and m respectively. Finally, we define the sets: 

T (n,m,j) — {t : t G T (n,m) and \t\ = j} 


and 

A (m,n,j) = {£ t~ f3i : t G T(n,m,j) and fa G P(n,m)}. 

The sets P(n,m ) are generated recursively as follows: 

P(n , m) = S U {op(a, b) : op G { , => , A, V} and a, b G S} , 


( 2 ) 
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where S = P(n — 1 , m) U Neg(P(n — 1 , m), Neg(X) = {~ a\a G X} and P(0, m) is the set 
that consists of the hrst m variables of P. 

Note that, given an order to the set of Boolean operators, we can define an enumeration 
of all the members of P based on its generation order , an order which is consistent among 
all the P(n , m) subsets for a given m. The order defined is equivalent to giving each P(n, m) 
set an array structure on which the hrst position correspond to the hrst element of P(n, m ) 
and f(n,m ) to the last, where / is defined as 

f(n, m) = | P(n, m)\. 

We call this array the propositions array. Given this order, we can represent each theory 
comprising T (n, m ) by a binary string of size f(n, m) on which the ith bit is 1 iff fa is in the 
theory. Following the previous idea, we can efficiently represent the members of T(n,m,j) 
by an array of j integers, where each integer denotes the number of Os present between 
each 1. Hence we can represent the theories t G T (n,m,j) by integer arrays of the form 
K = [k \,..., kj\. We call this the j -representation of the theory and denote it by tj<- 

Definition 3. If <f> G P(n,m) and <f> (jL P{n — 1 ,m) ; , then n is known as the syntactic 
complexity of 0 or depth of fa it is denoted by s(fa). 

Now let’s consider the following function: 

j 

gs(t K ) = ^2 k i- 

i=l 

The function gs gives us the degree of separation between the hrst element in the array 
of propositions ([0,...,0]) and tx- Moreover, there is an exponential number of theories that 
share a value for the function. We call such a set, defined by [g\ = {t\gs(t) — g}, a separation 
class and g the separation order of all the theories in the class. Recall that the position of 
each proposition depends on its order of generation. Hence the syntactic complexity of each 
theory is set by its order of separation. 

Definition 4. We define the syntactic complexity of a theory t, denoted as s(t), as 

s(t) = max{s(fa) : 0 G t}, 
where s(0) is the syntactic complexity of 0. 

Note that t G [s(t)]. 

0 n 

( 16m ) 

The size of the systems we are exploring grows by ~ 2 re , making an exhaustive 
exploration intractable. The methodology used therefore consists in sampling the space of 
valid propositional sentences. The propositions are then used as theorem candidates against 
subsets of the formulae used as theories. 
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In order to compute each sample set we build two sets: a sample set of theories denoted 
by T, composed of x number of theories in T (n,m,j), and the sample set of prospective 
theorems denoted by O, composed of \0\ numbers of propositions in P{n,m). Each set is 
randomly generated by, first, choosing a random list of numbers of the respective lengths 
between 1 and f(n,m). For the list O, each of these numbers represents the prospective 
theorems sampled (for each theory). For the list T, the numbers represent a separation class 
from which we choose a theory by assigning random numbers to each of the parameters of 
its j representation, with the condition that their sum is the value of the chosen class. The 
chosen lists are then rid of inconsistent theories for T and inconsistent propositions with 
respect to the first element for the list O. 

Afterwards, we use the lists obtained to compute a sample set of A (n,m,j). First, for 
each t e T we generate an additional number of theories of the form t U Oy, where Oy is a 
prefix of the list 0\ we call f = tUOo = tU0a base theory and tUOy a derived theory. Then, 
we pair all the theories generated with each of the propositions of Oj G O, called objectives, 
to form a list of cases of the form t U Oy b o r Afterwards, we remove the improvable cases 
using by exhaustively exploration the corresponding truth tables. 

It is important to note that we are generating a significant number of trivial cases when 
j' = j. Hence we expect at least close to ~ \0\ x \T\ instances of positive speed-up, depend¬ 
ing on the number of unprovable cases. 

Finally, we run an automatic theorem prover (ATP) and register the length of each of 
the proofs obtained, storing the shortest ones. However, we have no reason to believe that 
the use of an ATP system would give us a good approximation to the sparsity of proving 
speed-up. Therefore we define a speed-up function relative to each ATP: 

Definition 5. Let A be an ATP system, t b f3 a provable argument for A, t and t' two 
descriptions of [t] such that \t\ < \t'\. We define the speed-up delta relative to A of t b /3 
between t and t', or simply relative speed-up, as the function: 

PA(p)(t' b f3) 

D M p)(t b f3) 

where DA(p)(t b (3) is the shortest proof found by A for the argument t b j3. 

Definition 6. Let L be a formal system, A be an ATP system for L, t b /3 a provable 
argument for A, and t' a description for [t] such that t C t'. We call the function e t : N —» M 
a bound for A as a function of t if 



^A{p)it, t') > 0 + e t (\t'\). 

Now, since we do not have enough information about the existent sparsity of proving 
speed-up, we will define a necessary but possibly insufficient condition needed for an accept¬ 
able approximation: 
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Definition 7. We say that an argument t b f3 is trivial if f3 G t. An ATP system is pos¬ 
sibly normal for L if, for each t, €t(x) = 0, and if t b /3 is a non-trivial argument, then 
5A(p)(t,tU{P}) > 0 . 


(In lemma [8] we will show that normality is possible, although our example is far from 
ideal.) 

The mathematical structure used to store and analyze the results obtained is called the 
speed-up matrix. On this matrix each entry has assigned the value 

$i,j = &Oj (t(i,j),ti) 

where 

_ J U if t.j is a base theory. 

’ ' \ t : D(t b Oj) = mm D(th0j ){tk, otherwise. 

If ti b Oj is not a provable case, then the value of the entry is left undefined. 

A natural simplification of this matrix is the incidence speed-up matrix , on which we 
simply assign different discrete values to one of the following four cases: S hJ > 0, S ltJ = 0, 
Sij < 0 and Sij is undefined. 


Note that by design we expect a prevalence in both matrices of diagonal structures com¬ 
posed of cases of positive speed-up with a period of \0\. These structures correspond to the 
cases of trivial speed up included. 


3 Results 

To begin with, we performed more than 15 experiments using PROVER9. The following table 
resumes a select number of results: 


Exp. Num. 

Cases 

5 > 0 

Percentage 

5 < 0 

Ratio 

11 

5400 

606 

11.2% 

94 

6.44 

10 

6381 

704 

11.01% 

137 

5.138 

7 

4848 

389 

8.02% 

231 

1.683 

5 

5454 

426 

7.81% 

24 

17.75 

3 

11297 

856 

7.57% 

70 

12.228 


Figure 1: The results exhibit a varying percentage of negative and positive speed-up in¬ 
stances. It is important to note the presence of a significant number of negative speed-up 
instances and the irregular distribution found among the samples. 
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As the table shows, PROVER9 does not exhibit normal behavior as defined in[7| Further¬ 
more, as exemplified in Fig. [3j the speed-up matrix does not present the expected periodic 
diagonal speed-up instances. 
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Figure 2: A grayscale representation of the speed-up matrix obtained for experiment number 
11 using PROVER9. The columns correspond to each theory generated and the rows to the 
theorems. For visualization purposes, the matrix is divided into four parts and only the 
instances of positive speed-up are colored, the darker tones corresponding to higher speed¬ 
up values. 


Then, we present the Incidence Speed-up Matrix obtained from AProS under four dif¬ 
ferent conditions: the basic experiment (the four logical connectives and classical deduction) 
(3.1); exclusion of disjunction as a connective (3.3); the basic experiment with intuitionistic 
logic (3.2); intuitionistic logic while restricting to the negative fragment as in 3.3 (3.4). The 
same set of cases was used when possible, i.e. |3.1| with |3.2| and |3.3| with |3.4[ Also included 
is the matrix obtained for PROVER9 during experiment 3T It is important to note that 
we obtained no negative speed-up values. The matrix values are represented using a color 
scale, where the color white corresponds to no speed-up, blue to positive speed-up and red 
to negative speed-up. Grey corresponds to improvable cases or cases in which the time limit 
was reached. 


The four experiments yield similar behavior: although AProS does show periodic diag¬ 
onal structures it also exhibits a significant presence of negative speed-up instances, which 
makes the ATP otherwise than normal. 


It is important to note that a degree of clustering of negative speed-up instances is 
expected, since each delta is computed from the minimum proof length found for each previ¬ 
ously derived theory and current objective. It is arguable whether or not we are overcounting 
negative speed-up instances. 

Each speed-up matrix is divided into four parts for visualization purposes. Each of the 
entries’ values is represented using a four-color scale, where the color white corresponds to no 
speed-up, blue to positive speed up and red to negative speed-up. The columns correspond to 
each theory generated and the rows to each of the theorems. Grey corresponds to improvable 
cases or cases where a time limit was reached. 
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3.1 AProS speed-up incidence matrix with classical deduction and 
all four logical connectives 

Provable Cases: 5763. 

Positive S: 632, percentage: 10.97%. 

Negative S: 564, percentage: 9.78%. 


Figure 3: A color scale representation of the incidence speed-up matrix obtained for exper- 
3.1| (classical deduction and all four logical connectives) using AProS. The periodic 
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diagonal structures that correspond to the trivial speed-up instances are evident in this fig¬ 
ure, but it also manifests a significant presence of negative speed-up instances, which means 
that the AProS is not a normal ATP system. 


3.2 AProS speed-up incidence matrix with intuitionistic deduc¬ 
tion and all four logical connectives: 

Provable Cases: 5680. 

Positive 5: 646, percentage: 11.37.% 

Negative S: 537, percentage: 9.45%. 




Figure 4: A color scale representation of the incidence speed-up matrix compiled for experi¬ 
ment 3.2 (intuitionistic deduction and all four logical connectives) obtained from the same set 
of arguments used to compile the figure [3j The behavior is very similar to behavior observed 
in experiment |3.1| along with the significant presence of negative speed-up. We detected 


a small increase in the number of positive speed-up instances and a negligible decrease in 
negative speed-up cases. 







3.3 AProS speed-up incidence matrix with classical deduction and 
without disjunction 

Provable Cases: 6680. 

Positive <5: 899, percentage: 13.46 %. 

Negative 8: 484, percentage: 7.246%. 



Figure 5: A visual representation of the incidence speed-up matrix compiled for the exper¬ 
iment 3.3 (classical deduction without disjunction) divided into four parts for visualization 
purposes. It is important to note that the set of arguments employed for the previous exper¬ 
iments is incompatible with the parameters established for this case. Hence a new random 
set had to be generated. From the image we can see that the speed-up distribution does 
not differ notably from previous experiments, aside from the significantly lower incidence of 
undemonstrable arguments. 


3.4 AProS speed-up incidence matrix with intuitionistic deduc¬ 
tion and without disjunction 

Provable Cases: 6660. 

Positive 5: 862, percentage: 12.94%. 

Negative <5: 587, percentage: 8.81%. 
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Figure 6: A visual representation of the incidence speed-up matrix generated for experiment 
3.4| (intuitionistic deduction without disjunction) obtained from the same set of arguments 


used to compile the figure [5j We can see that the behavior is very similar to that observed 
in experiment |3.3|, along with the significant presence of negative speed-up. We detected 


a negligible decrease in the number of positive speed-up instances and a small increase in 
negative speed-up cases. 


3.5 Prover 9 incidence speed-up matrix without disjunction: 

Provable Cases: 6680. 

Positive S: 312, percentage: 4.67%. 

Negative S: 0, percentage: 0%. 
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Figure 7: A color scale representation of the incidence speed-up matrix obtained for experi¬ 
ment 3.1 (classical deduction with all four logical connectives) using Prover9. The image 
is divided into four parts for visualization purposes. As with figure [3] the absence of the 
predicted diagonal structures is conspicuous, but of greater importance is the total absence 
of instances of negative speed-up. 


3.6 Observations and Conclusions 

The main objective of this project was to undertake an empirical exploration of the prevalence 
and distribution of instances of positive speed-up found within the propositional calculus. In 
particular, two deduction systems for propositional logic were explored: natural deduction 
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and binary resolution, each of which was approximated by two automated proving systems, 
AProS and Prover 9. A necessary (but not sufficient) condition was proposed in order to 
decide the adequacy of these approximations (Def. [7j) . 

Given the speed-up matrices obtained, it is evident that neither AProS nor Prover 9 con¬ 
forms to the normality condition defined : 

Prover9 cannot detect trivial cases with regularity; instead of the expected periodic diag¬ 
onal patterns induced by the presence of instances of trivial speed-up, we find a number of 
vertical clusters of speed-up instances without a discernible regular distribution. This behav¬ 
ior is incompatible with the second condition of normality. We also found a non-negligible 
number of negative speed-up instances when a disjunction is included in the list of logical 
connectives. The presence of the disjunction seems to have little effect on the distribution 
of instances of positive speed-up. 

AProS shows an important number of instances of negative speed-up (slow-down). While 
AProS does not have problems detecting cases of trivial speed-up, the number of instances 
of negative speed-up is greater than in PROVER9. Furthermore, the presence and distribution 
of these instances is not significantly affected by the presence or absence of the disjunction, 
nor by the alternation between intuitionistic and classical deduction. 

We consider the observed behaviors as evidence of the computational complexity that 
the proposed condition of normality entails: discerning the usefulness of new information is 
intrinsically computationally complex. We formalize this in the following statements: 

Lemma 8. There is a normal prover. 

Proof. Given the argument t h /3, a brute force algorithm that, starting from the list t , 
simply searches for 0 while building the (infinite) tree of all possible logical derivations in a 
breadth-first search fashion, will always fold the proof of minimal length in the form of the 
selected branch. □ 

The expected computational time of this algorithm is of the order 0(q(D(t h /3)) D ^ h/3 ^), 
where q is a polynomial that depends on the number and structure of the derivation rules 
used. 

Theorem 9. Given a non-trivial argument t \- (3 and a non-empty set t' C P such that 
t C t!, deciding if 5p(t,t') > 0 is NP-Hard. 

Proof. Given the results found in nn we can say that, if P ^ NP, there is no polynomial 
time algorithm that can fold _D(0 h /3) (if it is polynomial with respect to |/3|). However, if 
we can decide 8p(t,t U A) > 0 in polynomial time then we can fold _D(0 h (3) in polynomial 
time: 

The algorithm we propose iterates the answer to (fg(0, {</>}) > 0 on each possible 0 deriva¬ 
tion from the list of chosen formulas D ( D starts with the list of axioms). The number of 
derived formulas is polynomial with respect to \D\. Each of the positive instances is added 
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to a list l. The formula in l that minimizes the proof length can be found in log(\l\)0(\l\) 
by pairwise comparison using the result of 8$({&}, {(pi, fij}) > 0 and 8p({4>j}, {</>;, 4>j}) > 0 
for each 4>i,4>j E l. Note that if both values are TRUE, then both formulas must be part of 
the smallest proof that contains any of the propositions, so we can choose to add just one 
formula and the other one will eventually be added on to the following iterations if it is part 
of the smallest demonstration. We add the selected expression to the list D. 


Following the stated procedure, we will eventually reach a trivial argument, finishing the 
demonstration of the tautology in the form of list D (the list contains a succession of formulas 
derived in order from t). Note that we consult a polynomial time algorithm a polynomial 
number of times over the size of a list that grows up to a polynomial size with respect to the 
input. Hence the algorithm finds the smallest proof in polynomial time. □ 

Given the demonstrated difficulty of the problem, we make the following conjecture: 


Conjecture 10. There is no 7iormal proving algorithm significantly faster than the brute 
force algorithm described in\l | 


In other words, we are proposing the existence of an ineluctable tradeoff between normal¬ 
ity and execution time of an automated proving algorithm. The conjecture H) also implies 
that, for AProS and PROVER9, the function e t is of exponential order for each t. 


Finally, in the context of an argument t h /3, we can say that a set A contains useful 
information if it induces (positive) speed-up Q and that the information is useless otherwise. 
With the experiment and the theorem [9] we have presented empirical and theoretical argu¬ 
ments as to why discerning the usefulness of new information for solving a specific problem 
is as hard as the problem itself. 


As for the differences found in the speed-up matrices for Prover 9 and AProS, we 
believe that these emerge mostly due to an initial syntactic analysis performed by AProS 
that allows it to detect trivial cases; the exception being when removing disjunction which 
results in no slow-down instances, although this change doesn’t seem to affect the positive 
speed-up distribution in a significant way. The conjecture IT suggests that, for AProS 
and PROVER9, the function e t is of exponential order for almost all t’s, and that both are 
within a linear constant between them, else we should be able to fold a shortcut to normality. 


Whenever the result presented in figure [7] is a counterexample to this statement and the 
conjecture 10 is an open question. We could argue that simplifying the formulas by removing 
number of logical connectives is doing the prover’s job. And, if we do restrict our space to 
simpler (yet complete) set of formulas, a stronger normality condition should be able to be 
defined as: 


Definition 11. A system is normal for L if there exist a polynomial time algorithm that 
calls ATP as an oracle for deciding 8p(t,t') > 0, with t, t' and fi as in theorem [9] 

1 This means that 6p(t, t U A) >0) 


12 






References 


[1] J. Joosten, F. Solcr-Toscano and H. Zenil, Program-size Versus Time Complexity, Speed¬ 
up and Slowdown Phenomena in Small Turing Machines, Int. Journ. of Unconventional 
Computing , special issue on Physics and Computation, vol. 7, no. 5, pp. 353-87, 2011. 

[2] H. Zenil, From Computer Runtimes to the Length of Proofs: With an Algorithmic 
Probabilistic Application to Waiting Times in Automatic Theorem Proving. In M.J. 
Dinneen, B. Khousainov, and A. Nies (Eds.), Computation, Physics and Beyond, WTCS 
2012, LNCS 7160, pp. 223-240, Springer, 2012. 

[3] J. Joosten, F. Soler-Toscano, H. Zenil, Speedup and Slowdown Phenomena in Turing 
Machines, Wolfram Demonstrations Project, Published: November 8, 2012. 

[4] A. M. Turing, On Computable Numbers, with an Application to the Entscheidungsprob- 
lem, Proceedings of the London Mathematical Society, 2 42: 230-65, 1937. 

[5] G. J. Chaitin, Algorithmic Information Theory, Cambridge Tracts in Theoretical Com¬ 
puter Science Volume 1, Cambridge Lhiiversity Press, 1987. 

[6] G. J. Chaitin, On the length of programs for computing finite binary sequences, Journal 
of the ACM, 13(4):547-569, 1966. 

[7] M. Hutter, Algorithmic complexity. Scholarpedia http://www.scholarpedia.org/ 
article/Algorithmic_complexity, 3(1):2573, 2009 

[8] K. Godel, On formally undecidable propositions of Principia Mathematica and related 
systems I in Solomon Feferman, Collected works, Vol. I. Oxford University Press: 144- 
195, 1986. 

[9] A. N. Kolmogorov, Three approaches to the quantitative definition of information. Prob¬ 
lems of Information and Transmission, 1(1):1—7, 1965. 

[10] R. J. Solomonoff, A formal theory of inductive inference: Parts 1 and 2. Information 
and Control, 7:1-22 and 224-254, 1964. 

[11] M. Alekhnovich, S. Buss, S. Moran, and T. Pitassi, Minimum Propositional Proof 
Length is NP-Hard to Linearly Approximate, The Journal of Symbolic Logic, Volume 
66: 171-191, 2001. 

[12] K. Kunen, Set Theory: An Introduction to Independence Proofs, Elsevier: 133-138, 
1980. 

[13] M. Huth, and M. Ryan, Logic in Computer Science: Modelling and reasoning about 
Systems, Cambridge University Press, UK, 2004. 

[14] J.E. Hopcroft, R. Motwani, and J.D. Ullman, Introduction to Automata Theory, Lan¬ 
guages, and Computation, Addison Wesley, Boston/San Francisco/New York: 368, 2007. 


13 



[15] B. Christoph, S. Geoff, Working with Automated Reasoning Tools, http://www.es. 
raiami.edu/~geoff/Courses/TPTPSYS/, 2008. 

[16] R. Cilibrasi, P.M.B. Vitanyi, Clustering by compression, IEEE Trans. Inform. Theory, 
51:12(2005), 1523-1545. 

[17] W. McCune, “PROVER9 and Mace 4”, http://www.cs.unm.edu/~mccune/Prover9, 
2005-2010. 

[18] W. Sieg, “AProS”, http://www.phil.cmu.edu/projects/apros/, 2006-2012. 

[19] W. Sieg, The AProS Project: Strategic Thinking & Computational Logic, Logic Journal 
of the IGPL , 15(4): pp. 359-368, 2007. 

[20] W. Sieg and J. Byrnes, Normal natural deduction proofs (in classical logic), Studia 
Logica 60, pp. 67-106, 1998. 

[21] M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide to the Theory 
of NP-Completeness, W.H. Freeman, 1979. 


14 



